Supplementary Material DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents

RNN Encoder 1 is responsible for encoding the past motion of the individual agent. The encoder is implemented with a temporal convolution layer (a.k.a, 1D convolution) that is followed by an RNN with GRU cells [1]. Before putting the past trajectory (Xi) to the temporal convolution layer, we subtract the last state value (present location) from xi,t at all time steps for a translation invariance. The temporal convolution layer has a kernel with 3 frames and 16 channel outputs. The GRU RNN has a 48 dimensional hidden vector, where the initial hidden vector is padded with 0.